Python-based Distributed Programming with Trickle

Trickle is a an extension to the Python programming language that provides explicit but simple mechanisms to write distributed scripts and programs. Trickle links together remote Python interpreters running on heterogeneous machines so that work can be deployed and results collected. A Trickle program interacts with remote interpreters by injecting functions or classes. Remote objects can be instantiated and invoked synchronously or asynchronously. Also, the injected code need not reside on the remote interpreters; code is dynamically transferred as needed. Trickle leverages off of Python’s list comprehensions and generators to simplify parallel invocation, result gathering, and dynamic scheduling. The Trickle run-time system uses a broadcast mechanism to find eligible Trickle virtual machines. Python programmers can immediately use Trickle to dispatch work to idle machines with minimal setup and easy to learn mechanisms. This paper describes the Trickle extension interface, its implementation, and presents an example application and its performance.